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Abstract: At the University of Twente, a course management system called “TeleTOP” has been 
implemented in all faculties the last 1.5 years. The paper describes the TeleTOP-system and the 
implementation process. To find out how teachers use their course environments in education, an 
analysis was made of 60 course environments. Results show, that these course environments are mainly 
used for the dissemination of information and less for interactive communication. Also, course 
environments help students prepare for learning, orientate on course content and practice, but are rarely 
used for giving feedback, monitoring and assessment. In the last section, tips and recommendations are 
given for improving the use of course environments. 
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Introduction 

Since the introduction of the world wide web, the web has been used in many ways to support education. The 
last three years, one particular use of the web has been very popular; the course management system. Different 
tools or platforms have been developed and implemented in universities all over the world. At the university of 
Twente, a choice was made for a home-made solution; the TeleTOP system. After successful use in one 
faculty, the university-wide implementation of TeleTOP started in the summer of 1999. Now, after 1.5 years of 
hard work, it is time to see what the results are. 

In this article, we present the results that can be achieved with the implementation of a course 
management system in a relatively short period of time. Also, by taking a closer look at the first 
implementation results we want to research what the “real” results are; to what extent are the implemented 
course environment really used in education? The third and most important goal of our research is to give 
examples and guidelines that instructional designers and teachers can use to improve the use of course 
environments in education. 
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TeleTOP, the Course Management System of the University of Twente 

As said in the introduction, TeleTOP is the course management system of the University of Twente. The goal 
of this system is to support and facilitate the learning process of students by supplying them with all the 
information they need to follow a course and with several options for communication with teachers and fellow 
students through the world wide web. As the University of Twente is a campus-based university, the course 
management system is used mainly to support on-campus learning. 

TeleTOP was developed in the beginning of the year 1998 at the Faculty of Educational Technology 
(Tielemans & Collis, 1999). The most important demands for the system were, that it should be consistent in 
terms of lay-out and structure, very easy to work with for both teachers and students, accessible through the 
WWW without any extra system requirements and based on a database-system. According to these wis hes, the 
development took place at the faculty of Educational Technology. Later in 1998 the Faculty started using the 
system in its courses. 

Because of the successful implementation at the faculty of Educational technology and also because 
of the lack flexibility in existing systems, the university board decided in 1999 to choose TeleTOP as course 
management system for the entire university. In the year 2000 the university wide implementation started on a 
voluntary basis. All technical facilities where supplied by the university and a team of instructional designers 
was available for educational support, but it was up to the faculties to decide if, and to what extent, they 
wanted to join the TeleTOP implementation. 
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Like most course management systems, TeleTOP is a course based system. For every course, an 
environment can be created. The teacher can s choose which functionalities he or she wants to use for his or her 
course. Figure 1 gives a schematic representation of the TeleTOP system. 
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Figure 1: Schematic representation of TeleTOP 



The core of every TeleTOP environment is “The Roster” (course schedule). This roster gives the student a very 
clear overview of the activities and materials in the course. In each TeleTOP site, the roster consists of 3 
columns that tell the student what to do before, during and after a session (see figure 2. The TeleTOP Roster). 
By clicking on a link in one of the roster cells, the student is taken to a page were he or she can find more 
detailed information, study materials and assignments related to that particular activity. 
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Figure 2: The TeleTOP Roster 



Implementation strategy and results 

The way TeleTOP has been implemented at the various faculties differs. In general we can distinguish three 

strategies used for the implementation of TeleTOP: 

1. Overall strategy: TeleTOP is implemented in all study phases of the program and all teachers are expected 
to use TeleTOP to support their courses; 

2. Phased strategy: The implementation starts in the first year of the program. Then, the system is gradually 
implemented into the other study-phases; 

3. Pilot strategy: The faculty starts with a few pilot courses. When the pilots prove to be successful, the 
implementation is expanded to one of the other strategies. 
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The implementation strategy is not prescribed by the university board, but is chosen by the faculty itself. For 
each faculty an instructional designer of the DINKEL Institute is available for support during the 
implementation process. 

As soon as the faculty board has decided to use TeleTOP in it's education and a choice for an 
implementation strategy is chosen, a demonstration of TeleTOP is organized for all teachers that will be 
involved. For the acceptance of TeleTOP by teachers it is proven to be important, that during this introduction, 
the faculty-director states clearly why and in what way TeleTOP will be used in the Faculty. 

After this plenary demonstration, in which a quick overview is given of the TeleTOP system and it’s 
possibilities, individual one-hour appointments are made with the teachers. These appointments can be divided 
in a technical instruction of TeleTOP and advice about the didactic use of the system. During this appointment 
the instructional designer and the teacher take a closer look at the course and decide which functionalities of 
TeleTOP (described in figure 1, §2.1.3 ) will be most suitable to support the course. After this one-hour 
introduction, the teacher is able to start filling the course environment with information and materials. For 
further support during the use of TeleTOP in their course teachers can contact the educational consultant, or 
take a look at the TeleTOP-supportsite (available on the WWW). This supportsite offers teachers technical as 
well as didactic tips for the use of TeleTOP in their course. 

Right now, after 1.5 years of implementation, more than 400 courses offered at the University of 
Twente have been implemented into the TeleTOP system. At this moment, all faculties have decided to 
participate in the TeleTOP implementation process. 



The Research 

Looking at the sections above, one might conclude, that the implementation of the TeleTOP system has been 
very successful until now. In one and a half year, more than 400 courses were implemented in TeleTOP and 
most of the teachers and students at this university are working with the system. However, what really matters 
is not how many course environments have been created and how many people are involved. If we really want 
to know how successful the implementation was, we have to look at what is happening inside these course 
environments to support the learning process. To find out what the real implementation results are, these are 
the research questions to be answered: 

a. For which goal(s) are the course environments used? 

b. How do the course environments support the learning processes of the students? 

To answer the first research question we identified which of the following three types of interactions 
were found in course environments; 

Information supply; the teacher supplies the students with information. This can be textual, but also 
graphical information, video and audio, etc. Information supply can be divided into two sub-categories, 
namely: 

■ Supply of organizational information : e.g. the course schedule tells the students when and where the 
classes take place. 

■ Supply of content-related information : e.g. in the course environment, the students can find 
documents in which quantum-mechanics are explained. 

Communication; the teacher communicates with students in an interactive way. This can for example be 
the answering of questions students send in, discussion statements, etc. Communication can be divided 
into two sub-categories: 

■ Organizational communication : e.g. the student asks the teacher when the assessment will take place. 

■ Content-related communication : e.g. the students have an online discussion about the solution of a 
math problem. 

Course work; students work on products such as reports, presentation, etc, within the course environment. 
This last category can be divided into: 

■ Individual course work ; one student works on a product and hands it in individually. 

■ Group-based course work ; students work together as a group on a project, report or something else, 
supported by the course environment and hand in their product as a group. 

The instrument used to answer the second question, is an existing educational model, based on instructional 
functions (Terlouw, 1997). Instructional functions describe which activities should take place in education to 
ensure that the desired learning processes take place. 
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The following main functions and sub -functions are defined: 

■ Preparatory functions: motivating, connecting to prerequisite knowledge, giving insight in learning goals, 
planning the learning process 

■ Executive functions : orientation on knowledge, orientation on skills and attitude, practice 

■ Regulative functions: monitoring progress and effort, feedback during practice, assessment, feedback after 
assessment 

The instructional functions mentioned above give no information about the medium or activity in which these 
are fulfilled. Within a course, some of these functions will be fulfilled by the teacher (e.g. the teacher gives the 
students feedback on their work), some by textual materials £.g. the book lets the student orientate on 
knowledge), etc. A course management system like TeleTOP in itself cannot fulfill any instructional functions. 
What we would like to know is which of these instructional functions are fulfilled through the TeleTOP course 
environment, using the environment as a carrier of information and tools. 

Because of time constraints, it was not possible to analyze all courses in TeleTOP. For this reason, a selection 
procedure was made to select 15 % of the existing TeleTOP environments. 10 faculties participated in the 
research. From each faculty, we selected 6 sites, which brings the total number of analyzed course 
environments at 60 (= 15% of 400). The course environments were randomly selected on their unique course 
code. 



Results 

In table one, we see in the first column the number of course environments (n = 60) in which a certain goal 
was addressed. In the second column, we see the same data in percentages. 



Goal 


# of course environments 


% of course environments 








Supply of organizational information (OI) 


60 


100.0 


Supply of content-related information (Cl) 


50 


83.3 








Organizational communication (OC) 


15 


25.0 


Content-related communication (CC) 


9 


15.0 








Individual course work (IW) 


25 


41.7 


Groupbased course work (GW) 


12 


20.0 



Table 1: Goals for using course environments 

In table two, we can see how the instructional functions were fulfilled through the 60 analyzed course 
environments. In column one, we can see, in what number of sites the sub-functions were found. In column 
two, in what percentage of the 60 analyzed sites at least one of the sub-functions was addressed. 



Instructional functions 


# functions 


% sites 








Preparatory functions 






Giving insight in learning goals 


37x 




Motivating 


29x 




Connecting to prerequisite knowledge 


15x 




Planning the learning process 


12x 




Percentage of sites in which at least one of the preparatory functions was found 




81.67 








Executive functions 






Orientation on knowledge 


48x 




Orientation on skills and attitude 


19x 




Practicing 


37x 




Percentage of sites in which at least one of the executive functions was found 




88.33 
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Regulative functions 






Giving guidance / monitoring progress and effort 


02x 




Feedback during practice 


21x 




Assessment 


03x 




Feedback after assessment 


Olx 




Percentage of sites in which at least one of the regulative functions was found 




20.00 



Table 2: Instructional functions fulfilled through the TeleTOP course environments 



Conclusions 

Looking at our research questions and the results, we can draw the following conclusions: 

For which goal(s) are the course environments used? 

In the course environments we analyzed, the emphasis is clearly on the dissemination of information from the 
teacher to the students. This information is not only organizational but also content-related. In the TeleTOP 
environments, teachers offer students lecture notes, articles, etc. This matches with the results of the university- 
wide evaluation of TeleTOP (Peters & Visser, 2001) where students indicate, that the most important benefit of 
TeleTOP is, that it provides access to information and study materials. In contrast to the first conclusion, not so 
many course environments are used to support two-way interaction between teachers and students. Content- 
related communication seems even more difficult to realize. This also corresponds with the results of the 
TeleTOP evaluation (Peters & Visser, 2001) in which teachers indicate that they do not perceive a functionality 
like discussion as very useful, because the discussions are seldom content-related and often not very serious. 
However, with this conclusion, we have to add, that there could be a lot of communication through lectures and 
e-mail, which is not visible in the TeleTOP environments. Finally, if a course environment is used to support 
course work, the emphasis is mostly on individual course work and less on group-work. 

How do the course environments support the learning processes of the students? 

The results show us, that TeleTOP environments are primarily used by teachers to prepare students for learning 
and helping them to practice and orientate on course content. More attention could be given to helping the 
students plan their learning and connecting to prerequisite knowledge. Remarkable is, that the amount of 
guidance, monitoring and feedback is still very poor. Even when students are asked to hand in course work, 
feedback is not often given through TeleTOP, but face to face or sometimes not at all. Finally, in almost all 
cases, assessment seems to take place outside of the course environments. 

From this research we conclude, that although a lot of activity and dissemination of information is 
taking place in TeleTOP, some possibilities of the system are not (or not sufficiently) exploited yet. During the 
first 1.5 years of the TeleTOP implementation, teachers had to put a lot of time and effort in getting to know 
the system and making their paper-materials available in a digital format. An earlier evaluation study at the 
Faculty of Educational Technology (Collis & Messing, 2001) shows the same results. Now that this first 
implementation round has been done, it is time to move to a second round in which the emphasis is on 
didactics and online interaction. Together with educational consultants, teachers can find ways to come to a 
better use of their course environments. 



Tips and recommendations 

In order to elaborate on our conclusions, we conclude with tips and recommendations that teachers and 
educational designers can use to improve the use of learning environments. Although this research was fully 
based on the situation at the University of Twente, we feel that the tips and recommendations here could also 
be used at other educational institutions using a course management system to support their courses, 
a. Alternatives for content-related information: content related information in a course environment often 
exists of lectures, notes or articles. In addition to this, as a teacher, you can: 

• make a link from the course environment to online scientific magazines 

• give a list of relevant, fun and interesting websites students can use while making their assignments 

• show examples of excellent course work from students of previous study-years 
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• ask students to search for content-related information and collect this in the course environment 

b. To stimulate two-way communication between teacher and students, one can: 

• tell students how often you are going to answer questions that students post in the course 
environments, so students know when they can expect an answer or reaction 

■ ask one student to fulfill the role of moderator in an online discussion or ask a student to actively 
stimulate the discussion by putting in statements 

■ stimulate the use of discussion by putting the first items in yourself and invite students to join 

■ stimulate students to ask questions, by putting a few frequently asked questions and answers from last 
year’s group in the course environment 

c. The problem with content -related communication often is, that it doesn’t occur because it is a separate 
activity that is not integrated in the course. Some recommendations to stimulate this: 

■ The teacher plays an active role in the communication 

■ Clear assignments in the course environments tell the students what the topic is and what contribution 
is expected of them 

■ Participation is assessed and rewarded 

■ During the face-to-face meetings, the outcomes of the online communications are used in one of the 
activities 

d. There are several ways to motivate students in a course environment: 

■ ask students to prepare questions for the next lecture and post them in the course environment 

■ give students an active role in the development of the course environment, by making them (as part of 
a course assignment) deliver content for the site 

■ clearly state what intentions you have with the use of the course environment in the course 

e. Helping students plan their learning process 

■ In the course schedule, instead of only indicating when the lecture are and what assignments have to 
be made, also indicate how much time a student should spend on each course activity 

f. The problem with feedback often is, that it is very time consuming to give written feedback to each 
student in a course environment. Some alternatives for face-to-face feedback: 

■ provide a model-answer, so students can compare their answers to this example 

■ use peer review; make students assess each other’s work 

■ make practice assignments optional; students who really want to practice will make the assignments 
and receive feedback. Students who feel they don’t need the practice can skip the assignments. 

Final note: To stimulate teachers to a better use of their course environments, it is essential, that practical tips 
and recommendations are disseminated into the faculties. Below, we list a few of the many ways to do this: 

■ Build a support environment in which teachers can find didactic tips and good practice examples. 

■ Give teachers access to the course environments of their colleagues, so they can see how other teachers fill 
their course environments and learn from each other’s successes and failures. 

■ Make a list of ‘example course environments’ available for the whole university on a prominent place and 
explain to teachers why these are considered ‘good examples’. 

■ Organize didactic workshops where teachers can work on improving their course environments with help 
of an educational consultant. 
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